Volumes:
Volume in HDFS:
hadoop fs -ls /user/dannap001c
drwx------ - dannap001c CHQCDC_Users 10 2014-08-28 15:41 /user/dannap001c
To mount/Unmount a volume using the MapR Control System:
1) In the Navigation pane, expand the MapR-FS group and click the Volumes view.
2) Select the checkbox beside the name of each volume you wish to mount.
Click the Mount/Unmount button
Some Question & Answers:
MapR provides volumes as a way to organize data and manage cluster
performance. A volume is a logical unit that allows you to apply policies to a
set of files, directories, and sub-volumes. A well-structured volume hierarchy
is an essential aspect of your cluster's performance. As your cluster grows,
keeping your volume hierarchy efficient maximizes your data's availability.
Without a volume structure in place, your cluster's performance will be
negatively affected.
Creating
a Volume
When creating a
volume, the only required parameters are the volume type (Standard,Local or Remote) and
the volume name. You can set the ownership, permissions, quotas, and other
parameters at the time of volume creation, or use the Volume Properties dialog
to set them later.By default, the root user
and the volume creator have full control permissions on the volume
Creating New Volume:
Creating New Volume:
Ex: New Volume for user divakar(dannap001c)
If you see any permission issue, create Volume from datanode cli user as mapr and change the owner and permissions accordingly
maprcli volume create -name user.dannap001c -path /user/dannap001c -topology
/data/user -quota 50G
hadoop fs -chown dannap001c /user/dannap001c
hadoop fs -chmod 750 /user/dannap001c
Volumes In MapR MCS:
.
Volume in HDFS:
hadoop fs -ls /user/dannap001c
drwx------ - dannap001c CHQCDC_Users 10 2014-08-28 15:41 /user/dannap001c
1) In the Navigation pane, expand the MapR-FS group and click the Volumes view.
2) Select the checkbox beside the name of each volume you wish to mount.
Click the Mount/Unmount button
When Should I Use Volumes?
You can use volumes to enforce disk usage limits,
set replication levels, establish ownership and accountability, and measure the
cost generated by different projects or departments.Create a volume for each
user, department, or project.You can mount volumes under other volumes to build
a structure that reflects the needs of your organization.The volume structure
defines how data is distributed across the nodes in your cluster.Create
multiple small volumes with shallow paths at the top of your cluster's volume
hierarchy to spread the load of access requests across the nodes.
Volume Mount:
Volume Mount:
Mounts
one or more specified volumes. Permissions required: mnt, fc, or a
Ex: Mount
the volume "test-volume" at the path "/test":
maprcli
volume mount -name test-volume -path /test
1) I'm trying to understand how I can figure out how "fill" a
volume is in cases where no quota has been defined for the volume; that is, the
-quota and -advisoryquota parameters are set to 0.In such cases, what
drives the volume running out of space?
There are probably three answers
to this depending on what angle you take to this question.
1) You can see [1] how much space
each volume uses on the MapR Control System (MCS). If you like, you can also
accumulate the space for multiple volumes by defining an accounting entity for
all of the volumes of interest.
2) A volume can run out of space
when the topology it lives in becomes full or when the cluster itself fills up.
Filling up the topology will cause an alarm and the volume will live partially
outside specified topology. Filling up the cluster is more serious and will
prevent writes. You can fill up a cluster the obvious way, but it can (does!)
also happen when you have a cluster that is 80% full and you have a rack power
failure that takes out >20% of your available space. If you don't fix that
within the default maintenance window your cluster will naturally fill up with
replicas.
3) A volume can also fill up its
available space for meta-data. There is no hard and fast limit for meta-data as
part of a volume, but since the first 64K of each file is kept in-line inside
directories, you may be consuming more of this than you might expect.
2) I want to create a
volume(MyVolume) but it can not mount the volume. it says this error: Failed to
mount volume MyVolume, Permission denied its permission is root and admin.how
can i create volume that comes to folder my cluster?
You need to be root to mount the
volume. Or please check in permissions whether the logged in user has mount
permissions or not. Grant mount permissions and you could mount it.
You should
not provide the mount location as an existing folder. The parent folder path
should be passed on to the mount location.
3) Is it possible to assign certain nodes to a volume?
I have some data nodes (older machines) that I want to use for long-term storage.
I would like to be able to direct older files to those nodes. What is the best way to do that?
Yes. You can absolutely do this
3) Is it possible to assign certain nodes to a volume?
I have some data nodes (older machines) that I want to use for long-term storage.
I would like to be able to direct older files to those nodes. What is the best way to do that?
Yes. You can absolutely do this
